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(57) Abstract 



A computer database contains visual and other information of an object scene from which a television monitor (25) 
or film display (27), is created by electronically sampling points of the object scene information in the computer memory 
(15, 19). Undesirable effects of aliasing are significantly reduced and substantially eliminated by pseudo- randomly distri- 
buting, in a particular manner, the occurrence of the point samples in space and time. Realistic depth of field is obtained 
in the images, corresponding to what is observed through a camera lens, by altering the sample point locations to simulate 
passing them through an optical aperture (93) in a pseudo-random distribution thereacross. Further, effects of illumina- 
tion, shadows, object reflection and object refraction are made more realistic by causing each sample point to pseudo-ran- 
domly select one of a predetermined number of possible ray directions. 
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PSEUDO-RANDOM POINT SAMPLING 
TECHNIQUES IN COMPUTER GRAPHICS 
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Background of the Invention 

This invention relates generally to -the act of. 
computer graphics, and more specifically to the field of 
point sampling of visual scene information for the 
purpose of reconstructing an image of the visual scene. 

One form of computer graphics that is becoming 
widely practiced is to develop the sequence of video image 
frames of a moving object scene from information of the 
scene that has been stored in a computer memory. The 
object scene database contains information of the visual 
characteristics of the object scene, such as color, as 
well as information of movement. The creator of a 
sequence of video frames then uses a computer to 
electronically assemble signals of each video frame from 
15 . the database in a manner that provides the views and 
movement of the object scene that is desired by the 
operator to be displayed. 

The electronic signal for each video frame is 
typically developed by electronic sampling of the object 
20 scene database. A separate set of digital signals is 
developed to represent the color and/or intensity of each 
pixel of a standard raster scanned video monitor, foe each 
video frame produced. Each pixel is thus the smallest 
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resolution element of the video display. The color 
and/or intensity of each pixel is determined by sampling 
the database information to determine the characteristics 
of the object scene at the location of a given pixel. 
Such sampling is generally done by averaging the object 
scene information over a certain portion of the area of 
the pixel, or, more commonly, to sample the information at 
one or more points within the pixel, usually in some form 
of a periodically repeating pattern. 

Recent developments in the field of computer 
graphics have been directed.to -increasing the realism of 
the resulting images. Progress has been made in more 
faithfully reproducing object textures, shadows, 
reflections and transparencies, for example. Much 
effort has been directed to the problem of aliasing, as 
well. Existing sampling techniques tend to generate 
video image frames having "alias" images; that is, images 
that appear to be real but which are not specified in the 
computer database. This is generally recognized as a 
characteristic of images formed through variously used 
point sampling techniques. 

Therefore, it is a general object of the 
"present invention to provide computer graphics techniques 
that further improve the realism of the resulting video 
image frames and the totality of video productions 
generated from computer database representations of an 
object scene. 

Summary of the Invention 

This and additional objects are accomplished by 
the present invention wherein, briefly and generally, the 
•object scene information in the computer database is 
sampled by points that are pseudo-randomly distributed in 
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one or several functions or dimensions. According to one 
aspect of the invention, the point samples are pseudo- 
randomly distributed in a particular manner across the 
video image plane being constructed. According to 
another aspect, the pseudo-random distribution of point 
samples is taken over the time that is occupied by the 
video image frame being constructed. This substantially 
reduces or eliminates the undesirable aliasing, both 
spatially and temporally. The distribution of samples 
over time also increases the realism of the video frame by 
adding the image blurring that would occur if the object 
scene was- - being • photographed according to usual 
techniques . 

According to another aspect of the present 
15 invention, a video frame is constructed to have a depth of 
field by sampling the data base as if the object scene 
represented by it is being viewed through a lens of a 
limited aperture that will view in focus only a limited 
depth of the object scene. The point samples are pseudo- 
tandomly directed over a defined lens aperture when 
sampling the database information. 

According to another specific aspect of the 
-present invention, reflective and transparent 
characteristics of an object are taken into account by 
recognizing the degree of diffusion that occurs at each 
sample point. A particular angle of reflection or 
refraction is pseudo-randomly selected for each sample 
point from a range of possible angles depending upon the 
object characteristics. This adds realism to the 
resultant image by recognizing the diffuse, blurry nature 
of reflections and translucency that is possessed by most 
real objects. 

According to yet another aspect of the present 
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invention, an intensity distribution is specified for a 
light source that is illuminating the object scene. A 
single light source ray is pseudo-randomly selected from 
the specified light source distribution, for .each sample 
point. This technique has the advantage of eliminating 
harsh shadows that result from existing techniques, 
further adding to the improved realism of the images, when 
a light source is only partially obscured. 

Additional objects, advantages and features of 
the various aspects of the present invention will become 
apparent from the description of its preferred 
embodiments, which description should be taken in 
conjunction with the accompanying drawings. 

Brief Description of the Drawings 

Figure 1 illustrates generally a computer 
system that is suitable for carrying out the various 
aspects of the present invention; 

Figure 2 illustrates one possible form of 
object scene information that is stored in the computer 
memories of Figure 1; 

Figures 3 and 4 illustrate two existing point 
-sampling techniques; 

Figures 5, 6 and 7 show three specific 
embodiments of the pseudo-random spatial techniques of 
the present invention; 

Figure 8 illustrates spatial aliasing of the 
prior art techniques of Figures 3 and 4; 

Figure 9 illustrates the improvement brought 
about by the pseudo-random point sampling techniques of 
the present invention; 

Figure 10 shows a Fourier transform of a 
periodically sampled signal; 
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Figure 11 shows a Fourier transform of a 
pseudo-randomly sampled signal; 

Figure 12 illustrates generally the 
distribution of the point samples over time;' 

Figures 13, 14, 15 and 16 illustrate several 
specific embodiments of the pseudo-random time sampling 
aspect of the present invention; 

Figure 17 illustrates generally computer 
database sampling by a given distribution of sample 
points on an image plane; 

Figure 18 shows a sampling technique that 
provides an image with a depth of field; 

Figure 19 is a ray tracing example for a single 
sample that includes the effects of reflection, 
1-5 refraction and light source distribution; 

Figures 20, 21 and 22 illustrate additional 
details of the example shown in Figure 19; and 

Figure 23 provides yet another application of 
the general techniques of the present invention. 
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Descrip tion, of the Preferred Embodiments 

Referring initially to Figure 1, a general 
'computer system as illustrated that is suitable for 
carrying out the various aspects of the present invention 
to be described in detail below. A common bus 11 is 
connected to a central processing unit (CPU) 13 and main 
memory 15. Also connected to the bus 11 is keyboard 17 
and a large amount of disk memory 19. Either a 
commercially available VAX-11/780 or Cray large computer 
system is satisfactory. A frame buffer 21 receives 
output information from the bus 11 and applies.it, through 
another bus 23, to a standard color television monitor 25 
or another peripheral 27 that writes the resulting image 
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frames dicectly onto film, or both. Additionally, an 
output device can simply be a videotape recorder of the 
standard type. 

Figure 2 illustrates the organization- of the 
information of the object scene that is maintained in 
memory in the computer system of Figure 1. There are many 
ways to store such information, one being selected for 
illustration in connection with, the present invention. 
This technique involves breaking down the object scene 
into components,- these elements being referred to herein 
as geometric primitives. One such geometric primitive is 
a polygon 31, for example, illustrated in Figure 2 within 
an overlay 33 that shows in dotted outline a few adjacent 
pixels of the resulting display. The resulting' display , 
of course, shows the color and intensity of the object 
scene within each pixel to be uniform, the size of the 
pixel being the resolution element size of display. The 
polygon represents portions of the object scene to be 
represented in a video frame. 

The information stored in the computer memory 
for each object polygon is as extensive as necessary for 
producing a particular quality video image frame. Its 
.position certainly must be a piece of that information, 
conveniently specified by the x, y and z coordinates. 
The x, x y and z coordinates of each of the corner points of 
the polygon are stored for each video frame to be 
constructed, as shown in Figure 2 with respect to the 
polygon 31. The "x" and "y" numbers represent, of 
course, the horizontal and vertical positions, 
respectively, of the points, while the "z" number 
specifies its distance behind the video frame (image 
plane) being constructed. 

In order to be able to sample movement of the 
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object scene that occurs in the time period of one image 
frame, a technique described in detail below, information 
is also maintained for each polygon of its movement during 
such time period. In Figure 2, a second position 31' of 
the same polygon is illustrated with its corner point 
coordinates being stored as incremental changes over that 
of their initial positions. The position shown for the 
polygon 31 is preferably, for example, that at the 
beginning of a video frame, while the position 31' is that 
at the end of the video frame. The polygon can also 
change its shape during this time period. 

Besides the positions of each object surface 
polygon being stored in the data base, certain visual 
characteristics are stored for each, as well.. These 
include separate red, green and blue color reflection 
signals, red, green and blue transparency signals, extent 
of light diffusion upon reflection, extent of light 
dispersion upon transmission through the surface, and 
similar characteristics. The use of these and others are 
explained below in connection with the techniques of the 
present invention. 

Referring to Figure 3, a commonly used 
•technique for determining the color and/or intensity of 
each pixel of the image frame is illustrated. The 
information in the computer database, in. this example 
that of the polygons illustrated in Figure 2, that is 
present in the space occupied by a particular pixel is 
determined for a plurality of points within the pixel. A 
large number of points are illustrated in Figure 3, being 
periodically distributed in both dimensions, but there 
are even some techniques that use only one or a very few 
sample points per pixel. The nature of the object scene 
in each such sample point is determined, and those 
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determinations ace combined in some manner, such as by 
weighted or unweighted averaging, in order to determine 
the color and intensity of that pixel of the image frame. 

Figure 4 illustrates a similar periodic point 
sampling technique, except that not all point samples are 
taken in each case. Rather, the full density of the 
periodic sampling pattern is employed only in regions of a 
pixel where changes in the object scene occur, such as 
represented by a line 35. This image dependent technique 
thus reduces the number of samples and the processing time 
cequired. 

But these and other periodic sampling 
techniques result in reconstructed images that include 
"aliases" of the real image to be displayed. Much effort 
has been directed to anti-aliasing techniques, one 
approach being to process the video signal obtained from a 
periodic pattern point sample technique in order to 
eliminate the aliasing effects of the technique. Others 
have suggested sampling in a non-periodic, dithered 
manner for a number of specific sampling applications. 
The techniques of the present invention include 
improvements to and new applications of such prior 
approaches. 

Three different specific pseudo-random 
sampling techniques are illustrated in Figures 5, 6 and 7, 
wherein a single pixel is illustrated and, for simplicity 
of illustration, only four point samples per pixel are 
described. However, an actual implementation would 
likely use sixteen, or even as many as sixty-four samples 
per pixel, if all of the aspects of the present invention 
are utilized. For other specific implementations, a 
lesser number of samples, such as one per pixel, could be 
utilized. But in any event, the pattern of point 
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samples, both within each pixel and across the face of the 
image frame in its entirety, are non-periodic, and form a 
non-rectangular and non-rectilinear grid pattern. 
Further, each selected sampling pattern may, 
alternatively, extend over an area of multiple pixels or 
only part of a pixel. But the examples described herein 
use a sampling area coincident to that of one pixel, for 
simplicity of explanation. 

Each of the embodiments of Figures 5, 6 and 7 
determines the location of the sample points within the 
pixel by first dividing the pixel into a number of non- 
overlapping areas equal to the number of sample points, in 
this case four. A sample point is confined within each 
such area, thus aiding in keeping the sample points spread 
!5 out. The four areas of the pixel are labeled in the 
Figures as numbers 41, 43, 4'5 and 47. The areas are shown 
to be rectangular but can be some other shape. 

In the embodiment of Figure 5, the location of 
the sample point for each of these four areas is pseudo- 
20 randomly determined. Ideally, the "random" numbers to be 
used to determine their locations are purely randomly, 
but since they are so determined by computer, there is 
some element of repetitiveness of sample position within 
t its defined area, although the distribution of locations 
of a large number of sample locations matches that of a 
random distribution. The most common way for a computer 
to generate the x,y coordinates of each sample point is to 
use a look-up table maintained in memory that has a list 
of numbers with a distribution being that of a random set 
30 of numbers. But the usual technique is for the computer 
to step through the table of numbers in sequence, so there 
are some repetitions since the table of numbers has finite 
length. However, the length of the list of numbers can be 
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quite large so that repetition does not occur for a 
significant number of sample points. But in order to 
adequately describe both a completely random selection of 
sample locations and one controlled by such a computer 
5 look-up table, the locations are referred to here in this 
description as "pseudo-random". 

In an implementation of the technique of Figure 
5, the same sample pattern is used on every pixel in a 
given image frame. It is preferable, however, to 
10 eliminate all periodicity of the sample pattern, 
including making sure that no two adjacent pixels have the 
same sample pattern, This, can be done by- using a- 
sufficiently long look-up table of random numbers. It is 
preferable to generate a sample pattern with "no two 
adjacent pixels (including those diagonally adjacent) 
having the same pattern, a result of the techniques shown 
in Figures 6 and 7. 

Referring to Figure 6, each of the four non- 
overlapping areas of the pixel illustrated has a 
reference point positioned at a fixed location in each, 
such as its middle. Each actual sample point location is 
then determined by the computer by adding a random 
positive or negative number to each of the reference 
point's x and y coordinates. These offset numbers are 
25 randomly determined, such as from the computer random 
number look-up table, and so repetition of the pattern 
would not occur for some very large number of pixels. 

Another application of the same offset 
technique is a combination of the techniques of Figures 5 
and 6, as shown in Figure 7. This is similar to that of 
Figure 5 and differs from that of Figure 6 by having its 
reference points distributed rather than fi«ed in the 
middle of the adjacent pixel areas. The reference point 
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pattern of the embodiment of Figure 7 may be the same for 
each pixel, but the actual point sample locations are 
determined by adding a positive or negative ' x,y 
coordinate offset increment to the coordinates of each 
reference point. For convenience, a limit is placed on 
the maximum offset of each, as. indicated by the dotted 
outline around each of the reference points of Figure 7. 
The sample points in the embodiment of Figure 6, however, 
can be located anywhere within its respective portion of 
the area of the pixel. 

By first defining non-overlapping -areas in 
which* a single sample point lies, bunching up of sample 
points is avoided. It can be visualized that if each of 
the four sample points could be positioned anywhere 
15 within the entire pixel, there would be occasions, 
because of - the random selection of those specific 
locations, where two or more of the sample points would be 
bunched together. Although defining a range of potential 
point sample locations to be within a non-overlapping 
area accomplishes this, there could obviously be some 
variations of this specific technique, such as by 
allowing the areas to overlap slightly, or some other 
variation, it may even cause no problem in particular 
•applications if the sample points are chosen in a manner 
that their bunching together does occur occasionally. 

Each of the specific techniques described with 
respect to Figures 5, 5 and 7 provides a picture sampled 
from a computer database that has fewer aliased images 
than if a periodic point sample distribution is utilized. 
The technique shown in Figure 5, wherein the same pattern 
is repeated for each pixel of the image frame, provides 
some improvement, but the techniques according to Figures 
6 and 7 are significantly better in reducing aliasing. 
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The technique of Figure 7 has been observed to be the best 
of the three because it has an additional advantage of 
being less noisy. 

Referring to Figure 8, an example of how an 
aliased image can be obtained and displayed is given. 
Figure 8(A) is a "picket fence" image of "slats" 51, 53, 
55, 57 and 59. This image is being sampled by a periodic 
distribution of points 61, 63, 65, 67 and 69, shown only 
in a single dimension for simplicity. Since the period 
of the sample points is greater than that of a periodic 
intensity variation of the image, all of those variations 
will not be faithfully reproduced. Figure 8(B) shows the 
image of a video display that is developed from the 
samples of Figure 8(A), region 71 being of one intensity 
15 and region 73 being of the other. " Of course, the image of 
Figure 8(B) is not a faithful reproduction of the image of 
Figure 8(A)." But since three of the sample points hit a 
portion of the image having one intensity and the other 
two a portion of the image having the other intensity, the 
detail of the other variations cannot be faithfully 
reproduced. The curve of Figure 8(C) represents the 
intensity variation of the image of Figure 8(A), the curve 
of Figure 8(D) being the sampling function, and the curve 
of Figure 8(E) illustrating the resulting image of Figure 
25 8(B). 

One way that has been suggested to avoid 
forming such alias images is to increase the number of 
sample points so that the detail can be captured. That is 
to say, increase the number of samples in order to 
increase the well-known Nyquist limit. But to use extra 
sample points for this increases the computational 
complexity and can never really solve the problem; it only 
reduces its appearance somewhat. No matter how many 
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samples are used, however, there will always be some 
situations of aliasing, particularly when the scene is 
changing. In this case, such a picket fence can show as a 
flashing black-and-white image over a large area, a very 
undesirable result. 

Referring to Figure 9, the effect of a randomly 
distributed pattern of sample points is illustrated. 
Figure 9(A) assumes the same "picket fence" image in the 
computer database, as with Figure 8(A). But the sample 
points in Figure 9(A) are distributed non-periodically so 
that the resulting image of Figure 9(B) appears to be gray 
rather than having large areas that are all white or all 
black. The image of Figure 9(B) appears gray since 
alternate portions of the image are black-and-white, 
rather than having large areas of each color as in Figure 
8(B). Further, as the point samples of Figure 9(A) are 
scanned relative to the "picket fence" image, there will 
be some noisy visual effect, similar to film grain noise, 
but one of considerably less annoyance than a large area 
flashing black or white. The noise level is controlled 
by the number of samples per unit area. 

Figures 10 and 11 show in the frequency domain 
the effect of periodic and stochastic point sampling, 
respectively, in both of Figures 10 and 11, curves (A) 
are the same, being an original signal, chosen to be a 
sine wave in the space domain. Curves (B) differ, 
however, in that Figure 10(B) shows the frequency 
distribution of a spatially periodic sampling pattern, 
while Figure 11(B) shows the frequency distribution of 
the ideal stochastic sampling pattern, in both cases, 
the sampling frequency is assumed to be below the Nyquist 
limit of the original signal, so will not be able to 
faithfully reproduce the original signal. But the 
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comparison of the curves of Figures 10 and 11 show the 
anti-aliasing effect of a random distribution. The 
spatial sampling distribution across the image is 
preferably chosen so that a Fourier transform of such a 
distribution over an infinite plane approximates a 
Poisson disk distribution, as shown in Figure 11(B). The 
primary characteristics of such a distribution include a 
very high level at zero frequency, a substantially zero 
magnitude to a certain frequency (both positive and 
negative), and then a substantially constant magnitude at 
higher frequencies. Except at zero frequency, the 
sampling function in the frequency domain (Figure 11(B)) 
is substantially continuous. Such a distribution .in the 
frequency domain provides the desired spatial position 
randomness and avoids bunching of the sample points. The 
techniques described with respect to Figures 5-7 
approximate such a distribution. 

The distribution (C) in each of Figures 10 and 
11 shows the sampled signal in each of those examples, the 
result of convolving the signal of curve (A) with the 
sampling distribution of curve (B). In the periodic 
spatial sample example of Figure 10, a number of 
extraneous spikes are obtained since each of the sampling 
spikes of Figure 10(B) is individually convolved with 
each of the spikes of the signal of Figure 10(A). Since 
the frequencies of the signal of Figure 10(A) are in 
excess of that of the sampling function of Figure 10(B), 
the sampled signal of Figure 10(C) is not a faithful 
ceproduction of that of the original signal, when the 
sampled signal of Figure 10(C) is displayed, it is in 
effect multiplied by a lowpass filter similar .to that of 
of Figure 10(D). The resultant sampled signal is shown 
in Figure 10(E), which is the portion of the signal of 
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Figure 10(C) which is within the band pass of the filter 
function of Figure 10(D). The signal indicated at Figure 
10(E) is capable of reconstructing alias images that bear 
little or no resemblance to that of the original signal 
which was sampled. 

The sampled signal of Figure 11(C) also does 
not correspond with the original signal of Figure 11( E ) 
but when multiplied by its filter characteristics of 
Figure 11(D), the resultant' sampled signal of Figure 
IKE) is uniform over the frequency range of the filter. 
This produces in an image white noise, which is much 
preferable to reconstructing an apparent genuine image 
that does not exist. 

The techniques described with respect to 
Figures 5-7 can also be utilized in a sampling system that 
modifies the sampling pattern in response to the content 
of the image information being sampled, so called 
adaptive sampling. For example, if ima ge changes or 
detail within a portion of a sampling area required it, 
the pattern of sample points can be repeated in such an 
area portion in reduced scale. 

According to another aspect of the present 
"invention, similar sampling techniques are employed over 
time in order to add realistic motion blur, such as exist 
m video and. film techniques. Referring initially to 
Figure 12, the example pixel of Figures 5-7 is indicated 
to have each of its four samples taken at different times 
tl, t2, t3 and t4, regardless of the specific technique 
used to spatially locate the point samples. These times 
are selected to be within an interval that corresponds to 
a typical shutter opening for video frame acquisition 
which these techniques are intended to simulate. 
Therefore, if there is movement of the objects during the 



20 



30 



WO 86/07646 



PCT/US86/01356 



16 



10 



interval of a single frame indicated in the computer 
database, then the resultant image of that frame 
reconstructed from the samples being taken of the 
database information will similarly show motion blur. 

In order to reduce or substantially eliminate 
temporal aliasing, the distribution in time of the 
samples over the frame interval is pseudo- randomly 
determined. Referring to Figure 13, a time line is given 
wherein four non-overlapping intervals of time are 
designated as boundaries for each of the four sample 
points to occur. A pseudo-random-seiection~of -the time" 
for each sample within each of these intervals is what is 
shown in Figure 13. The same time distribution in Figure 
13 could be used for each pixel of the image frame being 
15 constructed, but is preferable that the sample tijnes be 
different for at least each of immediately adjacent 
pixels, in order to maximize the anti-aliasing that is 
desired. Temporal aliasing can occur when changes occur 
in the scene, such as a flashing light, more rapidly than 
samples are being taken. It will also be recognized that 
the distribution in time * illustrated in Figure 13 
involves the same considerations as the spatial 
distribution described with respect to Figure 5. 

Similarly, Figures 14 and 15 illustrate psuedo- 
random temporal sampling that is carried out in the same 
way as the spatial sampling described with respect to 
Figures 6 and 7, respectively. In Figure 14, the time of 
each sample is chosen to be a pseudo-randomly determined 
offset from the center of the interval designated for each 
sample to occur. In Figure 15, a reference time is 
pseudo-randomly determined for each sample within its 
interval, and then the actual time for each sample is 
determined as a shift from this reference an amount that 
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least one sample point in a substantially different 
relative position within the boundaries of its pixel from 
that of all other neighboring pixels. 

11. The method according to claim 8 wherein 
said improvement additionally comprises locating a 
plurality of sample points within each pixel and wherein 
said computer data base contains information of spatial 
movement of the objects during a time period represented 
by said image frame, and further wherein the color and/or 
intensity of the objects is determined for each of the 
plurality of sample points within each pixel at one of a 
plurality of different instants during said image frame 
time period, thereby to show any motion blur of the object 
scene in the resulting image frame electronic signal. 

12. The method according to claim 8 wherein 
said improvement additionally comprises the steps of 
establishing the characteristics of an optical imaging 
system, including aperture size and focal plane relative 
to the objects of the scene, and determining the 
characteristics of the objects for each sample by taking 
into account the distance of the objects from the focal 
plane and the size of the lens aperture, whereby the image 
frame electronic signal contains information of the • 
objects with a certain depth of field as determined by the 
characteristics of the optical imaging system. 

13. The method according to claim 8 wherein 
said improvement additionally comprises the steps of 
determining, for the sample points individually, a range 
of angles of reflection and/or refraction of the object 
scene at such points, and pseudo-randomly selecting one 
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5. The improved method according to claim 1 
wherein said at least one stored parameter includes a 
range of angles of reflection from objects in the scene. 



6. The improved method according to claim 1 
wherein said at least one stored parameter includes a 
range of angles of refraction by objects in the scene • 

wherein 
range of 
objects 

8. In a method of forming an electronic signal 
of a video image frame that individually specifies the 
color and/or intensity of each of an array of pixels that 
forms said frame, wherein the color and/or intensity of 
each pixel is determined by sampling, at at least one 
point within a boundary of each of said pixels, data 
stored in a computer data base that specifies spatial 
locations and visual characteristics of objects in a 
scene for said image frame, the improvement comprising 
electronically positioning said sample points in a 
spatial distribution across said frame such that a 
Fourier transform of such a distribution over an infinite 
plane is substantially continuous in some regions. 



7. The improved method according to claim 1 
said at least one stored parameter includes a 
spatial intensity variations of illumination of 
in the scene. 



9. The improved method according to claim 8 
wherein the grid pattern includes a plurality of sample 
points in substantially all of the pixels of the frame. 

10. The method according to claim 8 wherein 
the improvement additionally comprises locating said at 



WO 86/07646 



PCT/US86/01356 



25 

IT IS CLAIMED; 

1. In a method of forming an electronic signal 
of a video image frame that individually specifies the 
color and/or intensity of each of an array of pixels that 
forms said frame, wherein the color and/or intensity of 
each pixel is determined by point sampling, at at least 
one point within a boundary of each of said pixels, data 
stored in . a computer data base that specifies various 
parameters relating to an object scene, the improvement 
wherein the sampling is electronically accomplished by a 
pseudo-random distribution of at least one of said stored 
parameters such that a Fourier transform of such a 
distribution over an infinite extent contains 
substantially continuous regions. 

2. The improved method according to claim 1 
wherein said at least one stored parameters includes 
location of objects in the scene relative to the image 
frame . 

3. The improved method according to claim 1 
wherein said at least one stored parameter includes an 
amount that the objects in the scene change during the 
time represented by said video image frame, whereby 
object blur of said object scene is represented. 

4. The improved method according to claim 1 
wherein said at least one stored parameter includes a 
range of ray paths representative of that created by an 
optical imaging system, whereby a depth of field of said 
object scene is represented. 
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sampling of Figures 14 and IS herein, depth of field of 
figure 18 herein, and secondary rays of Figures 19-22 for 
shadowing and reflection in. a special image case. The 
resultant images of Figures 3, 5 and 7 of the above- 
referenced published paper, were made on a Cray computer 
with the source code listing of Appendix A. 

Appendix B is a program that implements all of 
the aspects of the present invention for spherical 
objects and resulted in the images of Figures 4, 6 and 8 of 
the above-referenced published paper. 

These computer programs contain material in 
which a claim of copyright is made by Lucasf ilm, Ltd. , the 
assignee hereof. This assignee has no objection to the 
duplication of Appendices" A and B by photocopying and the 
15 like but reserves all other copyright rights therein. 

Although the various aspects of the present 
invention have been described with respect to various 
preferred embodiments thereof, it will be understood that 
the invention is entitled to protection within the full 
20 scope of the appended claims. 
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the broad extent of the techniques described above that 
are a part of the present invention. The techniques can 
be used to determine a center of mass of an object, an 
example of something that is desirable to be determined in 
the course of computer aided design (CAD). An object 141 
of Figure 23 has its surfaces determined by a pseudo- 
random distribution of sample points, shown to extend 
through the object in dotted outline. The pseudo-random 
nature of this sampling assures that the measurement will 
be made on the actual object 141 and not some alias image 
of it. 

The various techniques of the present invention 
have also been described by the inventors in a published 
paper, "Distributed Ray Tracing", .Computer Granhi^ . 
1> Vol. 18, No. 3, pages 137-145, July, 1984, which is 
incorporated herein by reference. This paper includes 
Photographs of images generated with the use of the 
various aspects of the present invention. The result of 
motion blur, as described with respect to Figures 12-16 
is shown in Figures 3, 6 and 8 of that paper. Computer 
generated images having a depth of field are shown in 
Figures 4 and 5 of that paper, having been made by the 
techniques described with respect to Figure 18 herein. 
Figure 7 of that paper illustrates the shadowing and 
reflection techniques of the present invention that were 
described with respect to Figures 19-22 above. 

Appendices A and B attached hereto are source 
code listings, in the C language, of computer programs 
implementing the various aspects of the invention 
described herein. They are part of a hidden surface 
algorithm. Appendix A is a general program that carries 
out the spatial sampling techniques of Figures 6 and 7 
herein, one of which is optionally selected, temporal 
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the possible refractive angles, another portion 125 of 
the object scene can then be determined which is 
intersected by the ray 121 and is partially visible 
through the object portion 113. 

In order to avoid sharp shadows, the realistic 
characteristics of an object scene illuminating light 
source 127 is taken into account. As shown in Figures 19 
and 22, the light source 127 has a finite extended 
dimension, and is not always a point as often assumed in 
present, computer graphics techniques. A ray 129 is 
traced from the illuminated surface 113 back to the source 
127 to see if . there is any other portion of . the object 
scene, such as the portion 131, that will cause a shadow 
to be cast on the surface 113. As shown in the example of 
Figure 19, the ray 129 will detect no such' shadow, but 
other possible ray directions, as shown in Figure 22, will 
be in the path of the object portion 131 and thus indicate 
that the object 113 is not illuminated by the source 127. 
The particular direction of the ray 129 is pseudo- 
randomly selected from those possible directions 
specified for the source 127, as shown in dotted outline 
in Figure 22. in the example of Figure 19, some of the 
rays will intersect the object portion 131 and some will 
not, resulting in soft, realistic shadows in the 
25 resulting image frame. 

It will be recognized that each of the 
secondary surfaces intersected by rays, such as the 
surfaces 117 and 125 of Figure 19, may also have 
reflective and translucent properties. The process is 
continued until such reflected or transparent images are 
so small in intensity as not to make any difference in the 
resulting image being constructed. 

Referring to Figure 23, an example is given of 
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The object surface portion 117 may be observed in the 
completed image frame as a reflection in the object scene 
portion 113. But stored in the computer database is a 
diffusive light spread of the surface 113, as indicated by 
dotted outline 119 and shown separately in Figure 20. it 
the characteristics of the surface 113 are specularly 
reflecting, such as occurs with a mirror, the spread of 
possible ray reflection angles will be limited to 
essentially one. But most objects have some degree of 
diffusion and will scatter light incident upon them. 
Therefore, each sample point ray is traced in a manner to 
select one of the possible reflection angles, thereby to 
result in a realistic blurry reflection from diffusely 
reflecting surfaces since subsequent rays . will be 
15 reflected off the surface 113 at one of the other possible 
angles shown in Figure 20. The possible ray reflection 
angles, as shown in Figures 19 and 20, are weighted in one 
direction, as is actually the case in diffusely 
reflecting surfaces. And, as before, the particular 
direction taken by any given ray 115 is pseudo-randomly 
selected from the possible reflection angles. 

The same consideration works in determining an 
angle of transmission of a ray 121 through the surface 
portion 113 if that surface portion is at all translucent. 
Assuming that it is, possible angles of refraction are 
stored in the computer database for that particular 
polygon, the distribution of such angles being indicated 
at 123 in Figure 19 and also shown in Figure 21. The 
spread of possible refractive angles depends, of course, 
30 on how diffuse the translucency is. Plain glass, for 
example, will have a very narrow range of -refractive 
angles, if not a single angle. And once the ray 121 is 
pseudo-randomly selected for a given sample point from 
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95 and 97 shown in Figure 18, are pseudo-randomly 
determined in the same manner as the earlier described 
pseudo-random determination of the spatial location and 
time of each sample point. 

Other unrealistic effects that result from the 
use of existing computer graphics techniques are sharp 
shadows, glossy reflections, and, if translucency of 
objects is taken into account at all, that also results in 
sharp images showing the translucent objects. This, of 
course, is not the real world of diffuse objects and 
extended . light sources, 'but are required simplifying - 
assumptions under previous algorithms in order to 
maintain within reason the complexity of the 
calculations. But the distributed techniques of the 
present invention can also be applied to these tasks, in a 
similar manner as described previously, to add these 
realistic considerations. Referring to Figure 19, a 
single ray 111. is traced from a single sample on the image 
plane (not shown) and interacts with the object scene in a 
manner specified by the characteristics of the light 
sources and object surfaces that are specified in the 
database. The techniques described with respect to 
Figure 19 are independent of the techniques described 
earlier, but, of course, maximum realism is obtained if 
all of these techniques are combined together. What is 
to be described with respect to Figure 19 occurs with each 
sample point of a particular image frame. 

The ray 111 is first determined to strike a 
surface 113 of the object scene, as specified by one of 
the polygons whose characteristics are stored in the 
computer database. If this part of the object scene 
surface is reflective, a reflective ray 115 is then traced 
until it intersects another object scene surface 117. 
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' frames. 

The example rays 89 and 91 of Figure 18 do not 
extend directly behind the image plane, as was described 
with respect to Figure 17, but rather are directed to 
intersect a simulated lens 93 at points 95 and 97, 
respectively. These rays then are directed again toward 
each other, under influence of refraction of the 
simulated lens. The rays intersect a focal plane 99 of 
the simulated optical system in the same pattern as exists 
on the image plane, as a result of defining the simulated 
optical system. The sample point rays 89 and 91 will then 
intersect polygons 101 and 103, respectively. Only 
polygons within the cone 105, shown in dotted outline, 
will be intersected with 'rays from sample points of the 
pixel 87, as defined by the characteristics of the optical 
system. Those polygons that are close to the focal plane 
99 will contribute to a focused reconstructed image, 
while those further removed from the focal plane 99 
contribute to an unfocused reconstructed image. In a 
computer software implementation of this technique, it 
has been found preferable to shift the x,y coordinates of 
the polygons an amount dependent upon their z distance 
from the focal plane 99 and the characteristics of the 
simulated optical system, and" then proceed with the 
sampling in a manner similar to that shown in Figure 17. 

But whatever specific implementation is 
carried out, the technique has the advantage of adding 
considerable realism to the simulated image at the time 
that the image is first formed by sampling the database. 
Intersection of sample rays with the simulated lens 99 
occurs over its entire defined aperture. In order to 
further reduce aliasing, the location of points of 
intersection of the rays with the lens, such as the points 
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location of these point samples has been determined by one 
of the techniques described above. Their individual 
rays are then projected, usually perpendicularly to the 
image plane, to determine the nearestmost polygons that 
are intersected by the rays at their selected time of 
sample. Much work has been done on such ray tracing 
techniques and involves a significant computer sort and 
matching of the x,y coordinates of the sample points with 
those of the polygons in the computer database at the 
instant designated for the taking of each sample. 
Usually, more than one _ polygon wilJL exist, at each x,y 
sampLe location, so the computer also determines from the 
"z" information of them which is the closest to the image- 
plane,, and that is then the one that provides the visual 
information (color, etc.) of the object scene at that 
point. All of the visual characteristics determined for 
each of the samples of a given pixel are then averaged in 
some manner to form a single visual characteristic for 
that pixel for display during that frame. 

Most computer graphics techniques show the 
entire object scene for each frame in focus, as if it was 
being viewed through a pinhole camera. This, of course, 
is not an accurate simulation of the real world of cameras 
and lenses, which have a limited depth of field. Depth of 
field can be taken into account by a ray tracing technique 
illustrated in Figure 18. a single pixel 87 has two 
sample points with rays 89 and 91 extending from them 
behind the image plane. The depth of field technique 
illustrated in Figure 18 is independent of the spatial and 
temporal sampling techniques described above, but it is 
preferable that those techniques be used in combination 
with the depth of field techniques being described in 
order to maximize the realism of the resulting image 
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performance of the following steps electronically: 

spatially dividing the area of said pixel into 
a plurality of non-overlapping areas, 

determining a nominal point location within 
each of said areas in a non-regular pattern, 

pseudo-randomly determining an offset of each 
such nominal point within each of said areas for each 
pixel within said image frame, 

determining from the computer data base the 
color and/or intensity of the closest of said objects at 
each offset point for each pixel of said frame, and 

combining the color and/or intensity 
information of the samples in each pixel/ thereby to 
determine a single color and/or intensity of each pixel. 

29. In a method of forming an image frame by 
individually controlling the color and/or intensity of 
each pixel in an array of pixels that forms said frame, 
wherein objects to be included in said image frame are 
represented by data stored in a computer data base that 
specifies spatial locations and visual characteristics of 
said objects for said image frame, a method of accessing 
the information of the computer data base for determining 
the color and/or intensity of each pixel, comprising 
performing the following steps electronically: 

spatially dividing the area of said pixel into 
a plurality of non-overlapping areas, 

determining a nominal point location in 
substantially the center of each of said areas, 

pseudo-randomly determining an offset of each 
nominal point within each of said areas for^each pixel 
within said image frame, 

determining from the computer data base the 
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27. In a method of forming an image frame by 
individually controlling the color and/or intensity of 
each pixel in an array of pixels that forms said frame, 
wherein objects to be included in said image frame are 
represented by data stored in a computer data base that 
specifies spatial locations and visual characteristics of 
said objects for said image frame, a method of accessing 
the information of the computer data base for determining 
the color and/or intensity of each pixel, comprising 
performance of the following steps electronically: 

spatially dividing the area of said pixel into 
a plurality of non-overlapping areas, 

pseudo-randomly positioning a sample point 
within substantially each of said areas, thereby to 
determine the position of a plurality of sample points for 
each pixel, 

determining from the computer data base the 
color and/or intensity of the closest of said objects at 
each of the plurality of sample points for each pixel of 
said frame, and 

combining the color and/or intensity 
information of the samples in each pixel, thereby to 
determine a single color and/or intensity of each pixel. 

28. In a method of forming an image frame by 
individually controlling the color and/or intensity of 
each pixel in an array of pixels that forms said frame, 
wherein objects to be included in said image frame are 
represented by data stored in a computer data base that 
specifies spatial locations and visual characteristics of 
said objects for said image frame, a method of accessing 
the information of the computer data base for determining 
the- color and/or intensity of each pixel, comprising 
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said improvement additionally comprises the steps of 
determining, for said sample points individually, a range 
of angles of reflection and/oc refraction of the object 
scene at each point, and pseudo-randomly selecting one 
such angle for each such sample point, whereby other 
portions of the object scene that are visible by 
reflection from or transparency through a sampled point 
of the object scene are determined* 

25. The method according to claim 17 wherein 
said improvement additionally comprises the steps of 
determining, for the sample points individually, a range 
of angles of rays extending from each such point to a 
source of illumination 'of the scene, pseudo-randomly 
selecting one such ray angle for each such point, and 
determining for each of said selected rays whether other 
objects are in the path of the ray, whereby penumbras are 
shown in the image frame, 

26. In a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of adjacent 
areas that forms said frame, wherein the color and/or 
intensity of each such area is determined by sampling, at 
a plurality of points in a certain pattern within a 
boundary of substantially every such area across said 
image frame, data stored in a computer data base that 
specifies spatial locations and visual characteristics of 
an object scene for said image frame, the improvement 
comprising the steps of electronically defining a 
plurality of substantially non-overlappinjg portions 
within such area and then locating each sample point 
within an individual area portion. 
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20. The method according to claim 17, wherein 
said electronic information contains information of 
spatial movement of the object scene during a time period 
represented by said image frame, and further wherein the 
color and/or intensity of the object is determined for 
each of the plurality of sample points within each frame 
area at one of a plurality of different instants during 
said image frame time period, thereby to show any motion 
blur of the object scene that occurs during the time 
period of said image frame. 

21. The method according to claim 20, wherein 
said one of a plurality of different instants of time is 
pseudo-randomly determined for saijd sample points. 

22. The method according to claim 17 wherein 
said improvement additionally comprises the steps of 
establishing the characteristics of an optical imaging 
system, including aperture size and focal plane relative 
to the objects of the scene, and determining the 
characteristics of tne objects for each sample by taking 
into account the distance of the objects from the focal 
plane and the size of the lens aperture, whereby the image 
frame electronic signal contains information of the 
o'bjects with a certain depth of field as determined by the 
characteristics of the optical imaging system. 

23. The method according to claim 22 wherein 
the step of determining the characteristics of the 
objects includes pseudo-randomly distributing the path of 
sample points through said lens aperture. 

24. The method according to claim 17 wherein 
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such angle for each such sample point, whereby other 
portions of the object scene that are visible by 
reflection from or transparency through a sampled point 
of the object scene are determined. 

14. The method according to claim 8 wherein 
said improvement additionally comprises the steps of 
determining, for the sample points individually, a range 
of angles of rays extending from each such point to a 
source of illumination of the scene, pseudo-randomly 
selecting one such ray angle for each such point, and 
determining for each of said selected rays whether other 
objects are in the path of the ray, whereby penumbras are 
shown in the image* frame. 

15. In a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of adjacent 
areas that forms said frame, wherein the color and/or 
intensity of each such area is determined by sampling, at 
a plurality of points in a certain pattern within a 
boundary of substantially every such area in said image 
frame, data stored in a computer data base that specifies 
spatial locations and visual characteristics of an object 
scene for said image frame, the improvement comprising 
electronically arranging said plurality of sample points 
in a manner that said certain pattern of substantially 
every such area is different than said certain pattern of 
substantially all other of such areas that are 
immediately adjacent thereto. 

16. The method according to claim 15 wherein 
said improvement includes making substantially all of 
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such areas coincident with pixels of the resulting video 
image frame. 

17. In a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of adjacent 
areas that forms said frame, wherein the color and/or 
intensity of each such area is determined by sampling, at 
a plurality of points in a certain pattern within a 
boundary of substantially every such area across said 
image frame, data stored in a computer data base that 
specifies spatial locations and visual characteristics of 
an object scene for said image frame, the improvement 
comprising electronically arranging said plurality of 
sample points within substantially every such area in a 
spatial distribution therein such that a Fourier 
transform of such a distribution over an infinite plane is 
substantially continuous in some regions. 

18. The method according to claim 17 wherein 
said improvement additionally comprises the step of 
causing said certain pattern of sample points within 
•substantially every one area to be different from that of 
substantially all its immediately adjacent areas. 

19. The method according to claim 17 wherein 
said improvement additionally comprises determining the 
certain pattern of sample points for each such area by a 
method comprising the steps of defining a plurality of 
substantially non-overlapping portions within such area 
and then locating each sample point within an individual 
area portion. 
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source of illumination of the scene, pseudo-randomly 
selecting one such ray angle for each such point, and 
determining for each of said selected rays whether other 
objects are in the path of the ray, whereby penumbras are 
shown in the image frame. 

48. In a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of pixels 
that forms said frame, wherein the color and/or intensity 
of each pixel is determined by sampling, at a plurality of 
points within a boundary of each of said pixels, data 
stored in a computer data base that specifies visual 
characteristics of an object scene for said image frame, 
the improvement comprising the steps of electronically 
determining, for the sample points individually, a range 
of angles of rays extending from each such point to a 
source of illumination of the scene, pseudo-randomly 
selecting one such angle for each such point, and 
determining for each of said selected rays whether other 
objects are in its path, whereby penumbras are shown in 
the image frame. 

49. A system for developing an electronic 
signal and displaying an image of an object scene 
therefrom, comprising: 

a computer memory database that specifies 
certain static and time varying visual characteristics of 
the object scene, 

means responsive to said computer memory for 
sampling information of the database to determine the 
visual characteristics of the object scene at a plurality 
of points* pseudo-randomly positioned across the object 
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sampled point of the object scene are determined, 

45. The method according to either of claims 
42 or 43, wherein said improvement additionally comprises 
the steps of determining, for the sample points 
individually, a range of angles of rays extending from 
each such point to a source of illumination of the scene, 
pseudo-randomly selecting one such ray angle for each 
such point, and determining for each of said selected rays 
whether other objects are in the path of the ray, whereby 
penumbras are shown in the image frame, 

46. in a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of pixels 
that forms said frame, wherein the color and/or intensity 
of each pixel is determined by sampling, at a plurality of 
points within a boundary of each of said pixels, data 
stored in a computer data base that specifies visual 
characteristics of an object scene for said image frame, 
the improvement comprising the steps of electronically 
determining, for the sample points individually, a range 
"of angles of reflection and/or refraction of the object 

scene at such points, and pseudo-randomly selecting one 
such angle for each such sample point, whereby other 
portions of the object scene that are visible by 
reflection from or transparency through a sampled point 
of the object scene are determined. 

47. The method according to claim 46 wherein 
said improvement additionally comprises the steps of 
determining, for the sample points individually, a range 
of angles of rays extending from each such point to a 
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the color and/or intensity of each of an array of pixels 
that forms said frame, wherein the color and/or intensity 
of each pixel is determined by sampling, at a plurality of 
points within a boundary of each of said pixels, data 
stored in a computer data base that specifies visual 
characteristics of an object scene for said image frame, 
the improvement comprising the steps of establishing the 
characteristics of an optical imaging system, including 
aperture size and focal plane relative to the objects of 
the scene, and electronically determining the visual 
characteristics of the object scene at each point sample 
by taking into account the distance of the objects from 
the focal plane and the size of the lens aperture, whereby 
the image frame electronic signal contains information of 
the objects with a certain depth of field as determined by 
the characteristics of the optical imaging system." 

43. The improved method according to claim 42 
wherein the step of determining the visual 
characteristics of the object scene for each point sample 
includes pseudo-randoraly determining the point on the 
lens from which the object scene is sampled within a field 
-of view of the object scene that is determined by the lens 
aperture and location of the focal plane. 

44. The method according to either of claims 
42 or 43, wherein said improvement additionally comprises 
the steps of determining, for the sample points 
individually, a range of angles of reflection and/or 
refraction of the object scene at such points, and pseudo- 
randomly selecting one such angle for each such sample 
point, whereby other portions of the object sc§ne that are 
visible by reflection from or transparency through a 
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comprises the steps of establishing the characteristics 
of an optical imaging system, including aperture size and 
focal plane relative to the objects of the scene, and 
determining the characteristics of the objects for each 
sample by taking into account the distance of the objects 
from the focal plane and the size of the lens aperture, 
whereby the image frame electronic signal contains 
information of the objects with a certain depth of field 
as determined by the characteristics of the optical 
imaging system, 

40. The method according to any of claims 34- 
36, inclusive, wherein said improvement additionally 
comprises the steps of determining, for the sample points 
individually, a range of angles of reflection and/or 
refraction of the object scene at such points, and pseudo- 
randomly selecting one such angle for each such sample 
point, whereby other portions of the object scene that are 
visible by reflection from or transparency through a 
sampled point of the object scene are determined. 

41. The method according to any of claims 34- 
36, inclusive, wherein said improvement additionally 

•comprises the steps of determining, for the sample points 
individually, a range of angles of rays extending from 
each such point to a source of illumination of the scene, 
pseudo-randomly selecting one such ray angle for each 
such point, and determining for each of said selected rays 
whether other objects are in the path of the ray, whereby 
penumbras are shown in the image frame. 

42. In a method of forming an electronic 
signal of a video image frame that individually specifies 
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instants during said time period, whereby any motion blur 
of the object scene is included in the resulting image 
frame electronic signal. 

35. The improved method according to claim 34 
wherein the step of sampling the data base at said 
plurality of points includes doing so pseudo-randoraly as 
a function of time for each such pixel and within said 
time period of said image frame, whereby the image frame 
signal has reduced aliasing in time. 

36. The improved method according to claim 34 
wherein the step of sampling the data base at said 
plurality of points includes the steps of defining an 
interval of time for taking a sample for each of said 
plurality of points for said pixel, and pseudo-randomly 
determining the instant of each such sample within its 
said defined interval of time, whereby the image frame 
signal has reduced aliasing in time. 

37. The improved method according to claim 36 
wherein the intervals of time defined for taking each 
sample are substantially non-overlapping. 

38. The improved method according to claim 36 
wherein the intervals of time defined for taking each 
sample are overlapped in a manner to increase the 
concentration of samples in the middle of the intervals of 
time relative to the concentration at their beginning and 
end. 

39. The method according to any of claims 34- 
36, inclusive, wherein said improvement additionally 
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29, inclusive, wherein said determining step includes a 
method comprising the steps of determining, for said 
sample points individually, a range of angles of 
reflection and/or refraction of the object scene at each 
point, and pseudo-randomly selecting one such angle for 
each such sample point, whereby other portions of the 
object scene that are visible' by reflection from or 
transparency through a sampled point of the object scene 
are determined. 

33. The method according to any of claims 27- 
29 wherein said improvement additionally comprises the 
steps of determining, for the sample points individually, 
a range of angles of rays extending from each such point 
to a source of illumination of the scene, pseudo-randomly 
selecting one such ray angle for each such point, and 
determining for each of said selected rays whether other 
objects are in the path of the ray, whereby penumbras are 
shown in the image frame, 

34. In a method of forming an electronic 
signal of a video image frame that individually specifies 
the color and/or intensity of each of an array of pixels 
that forms said frame, wherein the color and/or intensity 
of each pixel is determined by sampling, at a plurality of 
points within a boundary of each of said pixels, data 
stored in a computer data base that specifies visual 
characteristics of an object scene for said image frame, 
the improvement comprising the steps of including in said 
computer data base information of changes in the object 
scene visual characteristics that occur during the time 
period of said image frame, and electronically sampling 
the* data base at said plurality of points at different 
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